Distributed document co-authoring and processing

ABSTRACT

A method and a device are disclosed including plug-in software components that are integrated with document processing software suites. The plug-in software components provide a set of integrated interfaces for collaborative document processing in conjunction with multiple remote file, data, and application service providers. The set of interfaces enable coauthoring a document, document merging, discovering and displaying context-sensitive metadata on a software dashboard based on permissions associated with the metadata and/or a client computing device, caching, symmetric distributed document merge with the multiple service providers, and integrated search and insertion of multimedia data in documents, among others. The documents typically include, but are not limited to formatted text documents, spreadsheet documents, and slide presentation documents.

TECHNICAL FIELD

This application relates generally to document processing. Morespecifically, this application relates to distributed co-authoring andprocessing of a particular document via multiple third partycollaboration service providers and collecting and displaying metadatarelated to the particular document.

SUMMARY

In aspects of the present disclosures, a method of co-authoring adocument is disclosed including detecting a first set of changes made toa first version of a particular document on a first client computingdevice, detecting a second set of changes made to a second version ofthe particular document on a second client computing device, andcommunicating the first set of changes and the second set of changes tothe second client computing device and the first client computingdevice, respectively, via a third party collaboration service providerunassociated with a document processing suite used to edit theparticular document.

In further aspects of the present disclosures, a software plug-incomponent is disclosed, configured to be integrated with a documentprocessing suite, that when executed on a first client computing devicecauses the first client computing device to present an integrated userinterface within the document processing suite, the integrated userinterface configured to communicate with a third party collaborationservice provider not associated with the document processing suite.

In still further aspects of the disclosure, a method of co-authoring adocument is disclosed including dynamically discovering metadata relatedto a particular document being edited using a document processing suite,and displaying the discovered metadata in a dashboard area of a clientcomputing device executing the document processing suite.

BRIEF DESCRIPTION OF THE DRAWINGS

The drawings, when considered in connection with the followingdescription, are presented for the purpose of facilitating anunderstanding of the subject matter sought to be protected.

FIG. 1 shows an embodiment of a network computing environment whereinthe disclosure may be practiced;

FIG. 2 shows an embodiment of a computing device that may be used in thenetwork computing environment of FIG. 1;

FIG. 3 shows an example integrated document processing interface forediting documents usable with the computing device of FIG. 2;

FIG. 4A shows an example image search dialog usable with the integrateddocument processing interface of FIG. 3;

FIG. 4B shows an example web search dialog usable with the integrateddocument processing interface of FIG. 3;

FIG. 5 shows an example integrated document processing interface forspreadsheet applications usable with the computing device of FIG. 2;

FIG. 6 shows the example integrated document processing interface ofFIG. 3 including a dashboard with metadata;

FIG. 7A shows an example client-server computing environment withcommunication between the server and the dashboard on the client;

FIG. 7B shows an example integrated document processing interface forslide presentation applications usable with the computing device of FIG.2;

FIG. 8A shows an example system structure for document processing suiteapplications;

FIG. 8B shows an example interaction between multiple service providersand clients usable for collaboration;

FIG. 9 shows an example flow diagram for document processing;

FIG. 10 shows an example flow diagram for a word processing applicationmerge routine;

FIG. 11 shows an example flow diagram for a spreadsheet applicationmerge routine; and

FIG. 12 shows an example flow diagram for a slide presentationapplication merge routine.

DETAILED DESCRIPTION

While the present disclosure is described with reference to severalillustrative embodiments described herein, it should be clear that thepresent disclosure should not be limited to such embodiments. Therefore,the description of the embodiments provided herein is illustrative ofthe present disclosure and should not limit the scope of the disclosureas claimed. In addition, while following description referencesparticular office software suites like Microsoft Office® it will beappreciated that the disclosure may be used with other types of documentprocessing suites, such as Open Office®, Corel® and the like.

Briefly described, a device and a method are disclosed including plug-insoftware components that are integrated with document processingsoftware suites. The plug-in software components provide a set ofintegrated interfaces for collaborative document processing inconjunction with multiple remote file, data, and third party applicationservice providers. The set of interfaces enable coauthoring a document,document merging, discovering and displaying context-sensitive metadataon a software dashboard based on permissions associated with themetadata and/or a client computing device, caching, symmetric documentmerge with the multiple service providers, and integrated search andinsertion of multimedia data in documents, among others. The documentstypically include, but are not limited to formatted text documents,spreadsheet documents, and slide presentation documents. At least twobroad sets of features are provided by the present disclosure: one,functionality additional to what is traditionally associated withvarious document processing software suites such as co-authoring,image-based search, and offline access to remote files, and two,integration with third party collaboration servers that live in thecloud or on a LAN environment using file-related metadata, andintegrated comment processing.

Cloud computing refers to software applications and data that areaccessed remotely via the web in contrast to applications residing andaccessed locally on a user's computer. Some cloud computing solutionsenable users to share and work together on documents while others letusers store content and files on a location that is accessible from anycomputer. The integrated interfaces disclosed herein substantially bringcloud computing functionality into the users' local computer, allowingthe user to use office products' features while also seamlesslyincorporating new web functionality.

A number of companies produce document processing software products thattypically include a word processing program, a spreadsheet program, apresentation program, and an email and calendaring program, such asMicrosoft Office® (Word, PowerPoint, Excel and Outlook), Corel (WordPerfect Suite), Oracle Open Office®, and others. Throughout thisdisclosure the terms Office products, Office, document processingsuites, and the like include, but are not limited to word processingprograms such as Microsoft Word®, spreadsheet programs such as MicrosoftExcel®, electronic mailing (e-mail) and calendaring programs such asOutlook®, and slide presentation programs such as Microsoft PowerPoint®. Historically these programs ran and worked locally on the user'scomputer, using only that which was available on the local system, andstoring resultant work products on the local system. More recently, newfeatures have been added to some of these document processing productsthat allow a user to have limited interaction with the world wide web(web), for example, to use online (network accessible) storage, search,and similar limited features from within the respective products.

File, data, and third party application service providers use servers,such as Microsoft's SharePoint® server, to serve clients by providingstorage for data and functionalities that allow different clients tocollaborate and share data and information, for example, during thecourse of a project. Application service providers may include thirdparty collaboration servers/services unassociated with the documentprocessing suites used to edit documents. For example, some documentprocessing suites, such as Microsoft Office, are designed to work withcollaboration servers, such as SharePoint, but not with othercollaboration servers, and vice versa.

Various technologies, languages, and protocols are used to provide suchremote services to clients. For example, ASP (Active Service Pages), webparts (related to ASP), SOAP (Simple Object Access Protocol), XML(eXtensible Markup Language), JavaScript, C#, and other technologiesfurther described below may be used to interact with the web. SharePointWeb Parts are UI elements that support both customization andpersonalization. A Web Part is an ASP.NET server control designed to beedited and modified by knowledge workers. A Web Part works as acomponent of a SharePoint site that presents information pulled frommultiple data sources. With Web Parts, information dashboards can becreated on corporate portals and Web sites. Web Part pages allowpersonalization of information relevant to a user's needs by updatingthe value of specific Web Part properties.

With the ubiquity of users' internet access there is an ever increasingdemand for expanded services, functionality, online storage, sharingcapabilities, and the like. For example, some toolbars are availablethat is integrated with Microsoft Office® and allows the user to storedocuments online. However, these toolbars and the online storage theyprovide are limited in applicability, forces the user to use Microsoft®Office products, and does not provide the user with a full range ofdocument processing tools that may be available from different and/orcompeting service providers, such as real-time display of context-basedmeta data, collaborative authoring, full document merge capabilities,and the like.

Illustrative Operating Environment

FIG. 1 shows components of an illustrative environment in which thedisclosure may be practiced. Not all the shown components may berequired to practice the disclosure, and variations in the arrangementand type of the components may be made without departing from the spiritor scope of the disclosure. System 100 may include Local Area Networks(LAN) and Wide Area Networks (WAN) shown collectively as Network 106,wireless network 110, gateway 108 configured to connect remote and/ordifferent types of networks together, client computing devices 112-118,and server computing devices 102-104.

One embodiment of a computing device usable as one of client computingdevices 112-118 is described in more detail below with respect to FIG.2. Briefly, however, client computing devices 112-118 may includevirtually any device capable of receiving and sending a message over anetwork, such as wireless network 110, or the like. Such devices includeportable devices such as, cellular telephones, smart phones, displaypagers, radio frequency (RF) devices, music players, digital cameras,infrared (IR) devices, Personal Digital Assistants (PDAs), handheldcomputers, laptop computers, wearable computers, tablet computers,integrated devices combining one or more of the preceding devices, orthe like. Client device 112 may include virtually any computing devicethat typically connects using a wired communications medium such aspersonal computers, multiprocessor systems, microprocessor-based orprogrammable consumer electronics, network PCs, or the like. In oneembodiment, one or more of client devices 112-118 may also be configuredto operate over a wired and/or a wireless network.

Client devices 112-118 typically range widely in terms of capabilitiesand features. For example, a cell phone may have a numeric keypad and afew lines of monochrome LCD display on which only text may be displayed.In another example, a web-enabled client device may have a touchsensitive screen, a stylus, and several lines of color LCD display inwhich both text and graphic may be displayed.

A web-enabled client device may include a browser application that isconfigured to receive and to send web pages, web-based messages, or thelike. The browser application may be configured to receive and displaygraphic, text, multimedia, or the like, employing virtually any webbased language, including a wireless application protocol messages(WAP), or the like. In one embodiment, the browser application may beenabled to employ one or more of Handheld Device Markup Language (HDML),Wireless Markup Language (WML), WMLScript, JavaScript, StandardGeneralized Markup Language (SMGL), HyperText Markup Language (HTML),eXtensible Markup Language (XML), or the like, to display and sendinformation.

Client computing devices 12-118 also may include at least one otherclient application that is configured to receive content from anothercomputing device, including, without limit, server computing devices102-104. The client application may include a capability to provide andreceive textual content, multimedia information, or the like. The clientapplication may further provide information that identifies itself,including a type, capability, name, or the like. In one embodiment,client devices 112-118 may uniquely identify themselves through any of avariety of mechanisms, including a phone number, Mobile IdentificationNumber (MIN), an electronic serial number (ESN), mobile deviceidentifier, network address, such as IP (Internet Protocol) address,Media Access Control (MAC) layer identifier, or other identifier. Theidentifier may be provided in a message, or the like, sent to anothercomputing device.

Client computing devices 112-118 may also be configured to communicate amessage, such as through email, Short Message Service (SMS), MultimediaMessage Service (MMS), instant messaging (IM), internet relay chat(IRC), Mardam-Bey's IRC (mIRC), Jabber, or the like, to anothercomputing device. However, the present disclosure is not limited tothese message protocols, and virtually any other message protocol may beemployed.

Client devices 112-118 may further be configured to include a clientapplication that enables the user to log into a user account that may bemanaged by another computing device. Such user account, for example, maybe configured to enable the user to receive emails, send/receive IMmessages, SMS messages, access selected web pages, download scripts,applications, or a variety of other content, or perform a variety ofother actions over a network. However, managing of messages or otherwiseaccessing and/or downloading content, may also be performed withoutlogging into the user account. Thus, a user of client devices 112-118may employ any of a variety of client applications to access content,read web pages, receive/send messages, or the like. In one embodiment,for example, the user may employ a browser or other client applicationto access a web page hosted by a Web server implemented as servercomputing device 102. In one embodiment, messages received by clientcomputing devices 112-118 may be saved in non-volatile memory, such asflash and/or PCM, across communication sessions and/or between powercycles of client computing devices 112-118.

Wireless network 110 may be configured to couple client devices 114-118to network 106. Wireless network 110 may include any of a variety ofwireless sub-networks that may further overlay stand-alone ad-hocnetworks, and the like, to provide an infrastructure-oriented connectionfor client devices 114-118. Such sub-networks may include mesh networks,Wireless LAN (WLAN) networks, cellular networks, and the like. Wirelessnetwork 110 may further include an autonomous system of terminals,gateways, routers, and the like connected by wireless radio links, andthe like. These connectors may be configured to move freely and randomlyand organize themselves arbitrarily, such that the topology of wirelessnetwork 110 may change rapidly.

Wireless network 110 may further employ a plurality of accesstechnologies including 2nd (2G), 3rd (3G) generation radio access forcellular systems, WLAN, Wireless Router (WR) mesh, and the like. Accesstechnologies such as 2G, 3G, and future access networks may enable widearea coverage for mobile devices, such as client devices 114-118 withvarious degrees of mobility. For example, wireless network 110 mayenable a radio connection through a radio network access such as GlobalSystem for Mobil communication (GSM), General Packet Radio Services(GPRS), Enhanced Data GSM Environment (EDGE), WEDGE, Bluetooth, HighSpeed Downlink Packet Access (HSDPA), Universal MobileTelecommunications System (UMTS), Wi-Fi, Zigbee, Wideband Code DivisionMultiple Access (WCDMA), and the like. In essence, wireless network 110may include virtually any wireless communication mechanism by whichinformation may travel between client devices 102-104 and anothercomputing device, network, and the like.

Network 106 is configured to couple one or more servers depicted in FIG.1 as server computing devices 102-104 and their respective componentswith other computing devices, such as client device 112, and throughwireless network 110 to client devices 114-118. Network 106 is enabledto employ any form of computer readable media for communicatinginformation from one electronic device to another. Also, network 106 mayinclude the Internet in addition to local area networks (LANs), widearea networks (WANs), direct connections, such as through a universalserial bus (USB) port, other forms of computer-readable media, or anycombination thereof. On an interconnected set of LANs, including thosebased on differing architectures and protocols, a router acts as a linkbetween LANs, enabling messages to be sent from one to another.

Communication links within LANs typically include twisted wire pair orcoaxial cable, while communication links between networks may utilizeanalog telephone lines, full or fractional dedicated digital linesincluding T1, T2, T3, and T4, Integrated Services Digital Networks(ISDNs), Digital Subscriber Lines (DSLs), wireless links includingsatellite links, or other communications links known to those skilled inthe art. Furthermore, remote computers and other related electronicdevices could be remotely connected to either LANs or WANs via a modemand temporary telephone link. Network 106 may include any communicationmethod by which information may travel between computing devices.Additionally, communication media typically may enable transmission ofcomputer-readable instructions, data structures, program modules, orother types of content, virtually without limit. By way of example,communication media includes wired media such as twisted pair, coaxialcable, fiber optics, wave guides, and other wired media and wirelessmedia such as acoustic, RF, infrared, and other wireless media.

Illustrative Computing Device Configuration

FIG. 2 shows an illustrative computing device 200 that may represent anyone of the server and/or client computing devices shown in FIG. 1. Acomputing device represented by computing device 200 may include less ormore than all the components shown in FIG. 2 depending on thefunctionality needed. For example, a mobile computing device may includethe transceiver 236 and antenna 238, while a server computing device 102of FIG. 1 may not include these components. Those skilled in the artwill appreciate that the scope of integration of components of computingdevice 200 may be different from what is shown. As such, some of thecomponents of computing device 200 shown in FIG. 2 may be integratedtogether as one unit. For example, NIC 230 and transceiver 236 may beimplemented as an integrated unit. Additionally, different functions ofa single component may be separated and implemented across severalcomponents instead. For example, different functions of I/O processor220 may be separated into two or more processing units.

With continued reference to FIG. 2, computing device 200 includesoptical storage 202, Central Processing Unit (CPU) 204, memory module206, display interface 214, audio interface 216, input devices 218,Input/Output (I/O) processor 220, bus 222, non-volatile memory 224,various other interfaces 226-228, Network Interface Card (NIC) 320, harddisk 232, power supply 234, transceiver 236, antenna 238, hapticinterface 240, and Global Positioning System (GPS) unit 242. Memorymodule 206 may include software such as Operating System (OS) 208, and avariety of software application programs 210-212. Computing device 200may also include other components not shown in FIG. 2. For example,computing device 200 may further include an illuminator (for example, alight), graphic interface, and portable storage media such as USBdrives. Computing device 200 may also include other processing units,such as a math co-processor, graphics processor/accelerator, and aDigital Signal Processor (DSP).

Optical storage device 202 may include optical drives for using opticalmedia, such as CD (Compact Disc), DVD (Digital Video Disc), and thelike. Optical storage devices 202 may provide inexpensive ways forstoring information for archival and/or distribution purposes.

Central Processing Unit (CPU) 204 may be the main processor for softwareprogram execution in computing device 200. CPU 204 may represent one ormore processing units that obtain software instructions from memorymodule 206 and execute such instructions to carry out computationsand/or transfer data between various sources and destinations of data,such as hard disk 232, I/O processor 220, display interface 214, inputdevices 218, non-volatile memory 224, and the like.

Memory module 206 may include RAM (Random Access Memory), ROM (Read OnlyMemory), and other storage means, mapped to one addressable memoryspace. Memory module 206 illustrates one of many types of computerstorage media for storage of information such as computer readableinstructions, data structures, program modules or other data. Memorymodule 206 may store a basic input/output system (BIOS) for controllinglow-level operation of computing device 200. Memory module 206 may alsostore OS 208 for controlling the general operation of computing device200. It will be appreciated that OS 208 may include a general-purposeoperating system such as a version of UNIX, or LINUX™, or a specializedclient communication operating system such as Windows Mobile™, or theSymbian® operating system. OS 208 may, in turn, include or interfacewith a Java virtual machine (JVM) module that enables control ofhardware components and/or operating system operations via Javaapplication programs.

Memory module 206 may further include one or more distinct areas (byaddress space and/or other means), which can be utilized by computingdevice 200 to store, among other things, applications and/or other data.For example, one area of memory module 206 may be set aside and employedto store information that describes various capabilities of computingdevice 200, a device identifier, and the like. Such identificationinformation may then be provided to another device based on any of avariety of events, including being sent as part of a header during acommunication, sent upon request, or the like. One common softwareapplication is a browser program that is generally used to send/receiveinformation to/from a web server. In one embodiment, the browserapplication is enabled to employ Handheld Device Markup Language (HDML),Wireless Markup Language (WML), WMLScript, JavaScript, StandardGeneralized Markup Language (SMGL), HyperText Markup Language (HTML),eXtensible Markup Language (XML), and the like, to display and send amessage. However, any of a variety of other web based languages may alsobe employed. In one embodiment, using the browser application, a usermay view an article or other content on a web page with one or morehighlighted portions as target objects.

Display interface 214 may be coupled with a display unit (not shown),such as liquid crystal display (LCD), gas plasma, light emitting diode(LED), or any other type of display unit that may be used with computingdevice 200. Display units coupled with display interface 214 may alsoinclude a touch sensitive screen arranged to receive input from anobject such as a stylus or a digit from a human hand. Display interface214 may further include interface for other visual status indicators,such Light Emitting Diodes (LED), light arrays, and the like. Displayinterface 214 may include both hardware and software components. Forexample, display interface 214 may include a graphic accelerator forrendering graphic-intensive outputs on the display unit. In oneembodiment, display interface 214 may include software and/or firmwarecomponents that work in conjunction with CPU 204 to render graphicoutput on the display unit.

Audio interface 216 is arranged to produce and receive audio signalssuch as the sound of a human voice. For example, audio interface 216 maybe coupled to a speaker and microphone (not shown) to enablecommunication with a human operator, such as spoken commands, and/orgenerate an audio acknowledgement for some action.

Input devices 218 may include a variety of device types arranged toreceive input from a user, such as a keyboard, a keypad, a mouse, atouchpad, a touch-screen (described with respect to display interface214), a multi-touch screen, a microphone for spoken command input(describe with respect to audio interface 216), and the like.

I/O processor 220 is generally employed to handle transactions andcommunications with peripheral devices such as mass storage, network,input devices, display, and the like, which couple computing device 200with the external world. In small, low power computing devices, such assome mobile devices, functions of the I/O processor 220 may beintegrated with CPU 204 to reduce hardware cost and complexity. In oneembodiment, I/O processor 220 may the primary software interface withall other device and/or hardware interfaces, such as optical storage202, hard disk 232, interfaces 226-228, display interface 214, audiointerface 216, and input devices 218.

An electrical bus 222 internal to computing device 200 may be used tocouple various other hardware components, such as CPU 204, memory module206, I/O processor 220, and the like, to each other for transferringdata, instructions, status, and other similar information.

Non-volatile memory 224 may include memory built into computing device200, or portable storage medium, such as USB drives that may include PCMarrays, flash memory including NOR and NAND flash, pluggable hard drive,and the like. In one embodiment, portable storage medium may behavesimilarly to a disk drive. In another embodiment, portable storagemedium may present an interface different than a disk drive, forexample, a read-only interface used for loading/supplying data and/orsoftware.

Various other interfaces 226-228 may include other electrical and/oroptical interfaces for connecting to various hardware peripheral devicesand networks, such as IEEE 1394 also known as FireWire, Universal SerialBus (USB), Small Computer Serial Interface (SCSI), parallel printerinterface, Universal Synchronous Asynchronous Receiver Transmitter(USART), Video Graphics Array (VGA), Super VGA (SVGA), and the like.

Network Interface Card (NIC) 230 may include circuitry for couplingcomputing device 200 to one or more networks, and is generallyconstructed for use with one or more communication protocols andtechnologies including, but not limited to, Global System for Mobilecommunication (GSM), code division multiple access (CDMA), time divisionmultiple access (TDMA), user datagram protocol (UDP), transmissioncontrol protocol/Internet protocol (TCP/IP), SMS, general packet radioservice (GPRS), WAP, ultra wide band (UWB), IEEE 802.16 WorldwideInteroperability for Microwave Access (WiMax), SIP/RTP, Bluetooth,Wi-Fi, Zigbee, UMTS, HSDPA, WCDMA, WEDGE, or any of a variety of otherwired and/or wireless communication protocols.

Hard disk 232 is generally used as a mass storage device for computingdevice 200. In one embodiment, hard disk 232 may be a Ferro-magneticstack of one or more disks forming a disk drive embedded in or coupledto computing device 200. In another embodiment, hard drive 232 may beimplemented as a solid-state device configured to behave as a diskdrive, such as a flash-based hard drive. In yet another embodiment, harddrive 232 may be a remote storage accessible over network interface 230or another interface 226, but acting as a local hard drive. Thoseskilled in the art will appreciate that other technologies andconfigurations may be used to present a hard drive interface andfunctionality to computing device 200 without departing from the spiritof the present disclosure.

Power supply 234 provides power to computing device 200. A rechargeableor non-rechargeable battery may be used to provide power. The power mayalso be provided by an external power source, such as an AC adapter or apowered docking cradle that supplements and/or recharges a battery.

Transceiver 236 generally represents transmitter/receiver circuits forwired and/or wireless transmission and receipt of electronic data.Transceiver 236 may be a stand-alone module or be integrated with othermodules, such as NIC 230. Transceiver 236 may be coupled with one ormore antennas for wireless transmission of information.

Antenna 238 is generally used for wireless transmission of information,for example, in conjunction with transceiver 236, NIC 230, and/or GPS242. Antenna 238 may represent one or more different antennas that maybe coupled with different devices and tuned to different carrierfrequencies configured to communicate using corresponding protocolsand/or networks. Antenna 238 may be of various types, such asomni-directional, dipole, slot, helical, and the like.

Haptic interface 240 is configured to provide tactile feedback to a userof computing device 200. For example, the haptic interface may beemployed to vibrate computing device 200, or an input device coupled tocomputing device 200, such as a game controller, in a particular waywhen an event occurs, such as hitting an object with a car in a videogame.

Global Positioning System (GPS) unit 242 can determine the physicalcoordinates of computing device 200 on the surface of the Earth, whichtypically outputs a location as latitude and longitude values. GPS unit242 can also employ other geo-positioning mechanisms, including, but notlimited to, triangulation, assisted GPS (AGPS), E-OTD, CI, SAI, ETA, BSSor the like, to further determine the physical location of computingdevice 200 on the surface of the Earth. It is understood that underdifferent conditions, GPS unit 242 can determine a physical locationwithin millimeters for computing device 200. In other cases, thedetermined physical location may be less precise, such as within a meteror significantly greater distances. In one embodiment, however, a mobiledevice represented by computing device 200 may, through othercomponents, provide other information that may be employed to determinea physical location of the device, including for example, a MAC address.

FIG. 3 shows an example integrated document processing interface forediting documents usable with the computing device of FIG. 2. In variousembodiments, the integrated word processing interface 300 includes title302, set of menus 304, plug-in component 306 having set of buttons 308for activation of various functions including dashboard button 318,document 312 with title 310, text 314, and multimedia data 316.

In various embodiments, plug-in component 306 may appear as a toolbar, anew menu, or other common user interface components integrated with oneor more applications within the document processing suite. Those skilledin the art will appreciate that plug-in component 306 may be implementedas one or more modules, each module providing different functionalitiesfor the plug-in component as further detailed below. Applications withinthe document processing suite may include word processing, spreadsheet,email/calendar, and slide presentation programs, among others. Thoseskilled in the relevant arts will appreciate that plug-in or add-incomponents are generally software components that run within anotherhost program to add various capabilities, such as playing video clipplayers, encrypting content, messaging, and the like, within the hostapplication originally lacking such capabilities. Plug-in components areuseful at least because they may be deployed by third parties ontoexisting host applications after the release of such applications bytheir manufacturers. Plug-ins are usually installed by users afterinstallation of the host application. A host application may generallyprovide a service interface (or API—Application Programming Interface)for use by plug-in components to register themselves and exchange datawith the host application. Plug-ins may be implemented using DLLs(Dynamic Link Library) which are components that may be dynamicallyadded to a running program and provide various callable functions andcapabilities.

In various embodiments, plug-in component 306 generally providescapabilities which do not exist in the host program. For example,collaboration and image search functions may not be available in thehost applications, but are added by the plug-in.

A number of online service providers exist, such as third partycollaboration servers, which provide document management systems andcollaboration services for co-authoring documents. A user may sign upwith one or more of the online or web based document management andcollaboration platforms, or use an enterprise server (a server within alarge corporate environment or other organization, such as a governmentorganization) to create an online storage home directory for herdocuments. Once the user launches the integrated document processingsuite with the plug-in component 306 installed, the correspondingtoolbar, menus, and/or buttons allow the user to store, find, andretrieve her files from specific service providers with which the userhas registered. Each service provider may require the user to enter hercredentials, for example, via a dialog box, before allowing the useraccess to her files. Service providers are further described below withrespect to FIGS. 8A and 8B.

In various embodiments, plug-in component 306 may use the serviceprovider's API to specify the file and folder structure. Using such API,plug-in component 306 enables users to create new folders, rename filesand folders, change file access and processing (for example, read,write, delete, and the like) permissions, and take other actions enabledby the API. In effect, the distributed services provided by the serviceproviders in conjunction with the document processing suite appear tothe user as working on files and folders on the user's local computer.

In various embodiments, when the user clicks on or selects the “open”button on the set of buttons 308, the user is presented with a controlthat shows him the list of folders and files, which may be on the user'slocal computer or from the remote servers of the service providers. Theuser can navigate and filter the files based on specific metadata suchas file type, collaborators, size, folder and more, and thereafterselect a file to open. Once the user selects a file, the document isdownloaded and/or converted to an office format and opens in thecorresponding office application, such as word processing, spreadsheet,and the like.

In various embodiments, the metadata is sourced from a collaborationserver and/or local metadata. In other various embodiments, as part of adiscovery process, metadata is automatically and/or dynamicallydiscovered for and updated in a dashboard area further described belowwith respect to FIG. 6 and other figures. Discovered metadata by oneclient computing device may be communicated to remote servers and/orother client computing devices for update, display, or storage. Asstatus of different files, projects, resources, people, schedules, andother similar project related entities and information change, metadatadescribing such information and their status may be actively monitored,searched for, discovered, or otherwise ascertained and updated incorresponding sections of the dashboard area. Additional metadata notexisting in the dashboard area at a given time may later be addedautomatically as new information is discovered through the discoveryprocess of metadata described above. For example, in some embodiments,an autonomous software agent may roam various servers and sources ofinformation to find metadata related to a document or file being editedby a user. As the software agent discovers new information, the softwareagent communicates such information to the plug-in component of one ormore remote clients for display in the corresponding dashboard areas. Inother various embodiments, plug-in component 306 actively searches forand discovers new metadata, for example, by querying data stores onremote servers or other client computing devices.

In various embodiments, when the user clicks on or selects the “Save” or“Save as” button on the set of buttons 308, the user is presented with alist of folders and/or files, which may be on the user's local computeror from the remote servers of the service providers. The user mayprovide a filename, selects where she wants to save the file, and saveit. Plug-in component 306 typically uses the appropriate serviceprovider's API to upload and/or convert the file and store it on theremote server, in a manner that may be transparent to the user. As theuser modifies the document she is working on, she can occasionally savethe document. In some embodiments, plug-in component 306 may only savethe changes (delta) to the user's document made since last save on theremote server to save time and resources. In other embodiments, eachsave may result in a complete transfer and saving of the document beingedited.

In various embodiments, plug-in component 306 may provide a documentsearch button, which the user may use to find documents both locally andon remote service providers' servers. In various embodiments, plug-incomponent 306 integrates and provides document search functionality forfinding documents stored on remote web based collaboration platformsoperated by service providers. The search function may allow users toretrieve files by keyword search, URL (Uniform Resource Locator),server, directory pathname, database relational queries, and the like.Upon locating the desired file, plug-in component 306 downloads the filefrom the remote server to the user's computer for processing. Image andweb search capabilities are described below with respect to FIGS. 4A and4B.

In various embodiments, the “Collaboration” button in the set of buttons308 may be selected by the user to collaborate on or co-author aparticular document with other remotely located people. In oneembodiment, the user may use the “Settings” button to add names, emails,and permission levels for people with whom the user wants to grantpermission to access and/or collaborate on the particular document. Inother embodiments, plug-in component 306 may enable collaborationbetween multiple users by using permissions associated with a fileintended for coauthoring. When a user saves the file on the serviceprovider's remote server, the user may add the emails and names of thepeople with whom he wishes to collaborate in a Save dialog box. Once thefile is saved, plug-in component 306 may utilize the service provider'sAPI to associate collaboration permissions with the file according tothe user's specification. After the file is saved on the remote server,the Collaborate button allows the user to collaborate with the peoplewho have been given permission to access the file. In other embodiments,the other users who have been given permission by a different user tocollaborate on a particular file will be notified or otherwise detectthat they have access to the particular file for collaboration. Forexample, a file collaboration icon may be added to the file directorylisting of the particular file, which is visible to the other users toindicate that the other users now have collaboration permission on theparticular file.

In some embodiments, different metadata are displayed on differentcollaborating users' local dashboard areas depending on permissionsettings, user's preference and/or selection of metadata desired to bedisplayed, user's function in the collaboration project, system/networkthroughput and/or efficiency, cost of data transmission, or otherconsiderations. For example, a junior author and/or his client devicemay be granted permission to view a limited set of metadata associatedwith the particular document being edited, while a senior author may begranted additional privileges for access to the metadata. Such metadatapermissions are generally associated with both the metadata and theclient computing device of a particular user/author.

In various embodiments, co-authoring is platform-agnostic and may beused with any platform, such as SharePoint or other similar platforms,rather than being specific to a particular platform. In other variousembodiments, multiple users participating in co-authoring a particulardocument, each may use the same or different document processing suites.For example, one user may use Microsoft Office with a correspondingplug-in component 306, while another user may use Open Office with acorresponding plug-in component 306 to author the same document.

In various embodiments, the “Email” button in the set of buttons 308 maybe used to launch an email dialog box for checking, sending, andreceiving emails. In one embodiment, selecting the Email button launchesthe email/calendar program of the document processing suite, while inother embodiments, a limited email interface, for example, in the formof a dialog box running within the currently used application of thedocument processing suite (as opposed to launching a distinct officeapplication, such as an email/calendar application), may be presented tothe user for basic email activities.

In various embodiments, the “Web” button in the set of buttons 308 maybe selected to launch a separate browser program. In some embodiments,the browser may be a full function browser hosted within the documentprocessing suite, while in other embodiments, a limited browser dialogbox may be presented for basic browsing without other functions such asapplet hosting, script support, and the like. Browser functions arefurther described with respect to FIG. 4B.

The “Search” button in the set of buttons 308 may be selected to useexpanded search capabilities include image searches and web searches.Additionally, third party content providers may integrate their searchservices with plug-in component 306 search functionality, as is morefully described below.

FIG. 4A shows an example image search dialog usable with the integrateddocument processing interface of FIG. 3. In various embodiments, imagesearch dialog box 400 includes an image search title 402, search inputfield 404, search button 406, an image format pick list 408, an advancedsearch button 410, and a result section 412 including returned images414 each having a copy button 416 and an insert button 418.

An image search is different from text search or other web search inthat the returned results are generally in the form of images. Forexample, a web search with keyword “dog” will return results thattypically include text articles, pet shops, dog foods, veterinarians,may be some images of dogs, and the like. But, an image search with thesame keyword will typically return only images of dogs. In variousembodiments, image search dialog box 400 may pop up for user uponselection of search button from the plug-in 306 (FIG. 3) set of buttons308.

Existing office software applications provide limited searchcapabilities, typically embedded in the office software and/or from alimited number of websites with which the office software communicatefor obtaining additional data. This closed or limited serviceenvironment limits the user accordingly in what additional data he mayobtain from within the office software environment. In variousembodiments, the search function of plug-in component 306 (FIG. 3)searches the web for images using commercial search engines, such asGoogle® and Yahoo®. Search engine API may be used to filter searchresults according to intellectual property rights, such as copyright andtrademark. As such, the search functionality of plug-in component 306makes it unnecessary for the user to open a separate browser applicationdistinct from the office suite to search the web, obtain results, andthen copy and insert such results into his working documents within theoffice suite, thus, saving time and providing convenience for the user.

In various embodiments, an image search generally starts with activatingthe search button in the set of buttons 308 (FIG. 3). Image searchdialog 400 is presented to the user to input his search query intosearch input field 404. The user may select image format, for examplejpeg, bitmap, tiff, and the like, from pick list 408. The user may alsoselect advanced search button 410 to view additional input fields and/orother pick options and settings to be used as criteria to furtherrestrict the search. For example, in the advanced search settings, theuser may include or exclude a date range, a particular source, imagesize, image resolution, and the like. The search query thus entered bythe user is passed on to the appropriate search engine, such as Google,Yahoo, and the like, to execute and return the results.

Once the results are retrieved by the search engine, the results arepassed back to image search dialog of the plug-in component 306, forexample, using the search engine's API to render the results in asoftware control component, such as an ActiveX control, a Java applet,and the like, that displays a gallery of thumb nail images (smallpreview images). The results may include appropriate metadata about eachof the images, such as date of creation, author/creator, legal rights,resolution, format, and the like. The metadata may be used, eitherautomatically or by the user, to filter, sort, arrange, group, orotherwise manipulate the results into a final selection of images. Invarious embodiments, the user may browse the images and copy and insert,using buttons 416 and/or 418, one or more of the finally selected imagesinto the document the user is editing. Thus, the user is enabled tosearch the web from within the document processing suite environment,without launching other applications like browsers, for desired imagesand insert the search results in his documents.

FIG. 4B shows an example web search dialog usable with the integrateddocument processing interface of FIG. 3. In various embodiments, websearch dialog box 430 includes web search title 432, search input field434, advanced search button 436, results section 440 possibly showing aweb page 438 including contents that may be selected by the user asindicated by selected portion 442.

In various embodiments, in a manner similar to image search describedabove with respect to FIG. 4A, the search function of plug-in component306 (FIG. 3) searches the web for relevant results using commercialsearch engines, such as Google® and Yahoo®. Search engine API may beused to filter search results according to intellectual property rights,such as copyright and trademark. As such, the search functionality ofplug-in component 306 makes it unnecessary for the user to open aseparate browser application distinct from the office suite to searchthe web, obtain results, and then copy and insert such results into hisworking documents within the office suite, thus, saving time andproviding convenience for the user.

In various embodiments, a web search generally starts with activatingthe search button in the set of buttons 308 (FIG. 3). Web search dialog430 is presented to the user to input his search query into search inputfield 434. The user may select advanced search button 436 to viewadditional input fields and/or other pick options and settings to beused as criteria to further restrict the search. For example, in theadvanced search settings, the user may include or exclude a date range,a particular source, type of articles or web pages, age of web pages,and the like. The search query thus entered by the user is passed on tothe appropriate search engine, such as Google, Yahoo, and the like, toexecute and return the results.

Once the results are retrieved by the search engine, the results arepassed back to image search dialog of the plug-in component 306, forexample, using the search engine's API to render the results in asoftware control component, such as an ActiveX control, a Java applet,and the like, that displays the results. The results may includeappropriate metadata about each of the items in the results, such asdate of creation, author/creator, legal rights, age of web page, URL,relevance score, and the like. The metadata may be used, eitherautomatically or by the user, to filter, sort, arrange, group, orotherwise manipulate the results into a final selection of items. Invarious embodiments, the user may browse the results and select an item,for example, by double-clicking on the item, for full display. Invarious embodiments, the results selected for full display may bedisplayed as full web pages as rendered on an independent browserapplication, but within the computing environment of the plug-incomponent 306 (FIG. 3). The user may select portions of the displayedweb page, for example, selected portion 442, for copying and/or pastinginto his document. Thus, the user is enabled to search the web fordesired results and web pages from within the document processing suiteenvironment, without launching other applications like browsers, andinsert selected portions of the search results into his documents.

In various embodiments, the search functionality of plug-in component306 may be extensible to enable a third party provider, such as acontent provider, a search engine provider, or an independent softwarevendor (ISV), to integrate its search features with the plug-incomponent search functionality. With such integration, the third partyprovider may include its content into the search results of the plug-incomponent's search queries. In various embodiments, an extension APIassociated with the plug-in component 306 may enable the third partyproviders to include their content with the search results. Such thirdparty content includes, but is not limited to images and other types ofcontent, such as documents, web pages, document templates, and the like.

Conversely, in some embodiments, documents authored or modified in adocument processing suite integrated with plug-in component 306, maysave such documents on web based, third party provider platforms, suchas Google Apps, Google Docs, Cisco webex, IBM lotusLive, EMC, and thelike. This way, plug-in component 306 integrates seamlessly betweenoffice applications and web content management or document managementand collaboration platforms. Once the user's documents are saved on thethird party provider's servers, plug-in component 306 enables users toretrieve the saved files over a computer network and open them directlyin any of the office programs as if the files were saved locally on theuser's computer.

FIG. 5 shows an example integrated document processing interface forspreadsheet applications usable with the computing device of FIG. 2. Invarious embodiments, spreadsheet application 500 of the documentprocessing suite includes 502, tabs or menus 504, plug-in component 506(similar to plug-in component 306 of FIG. 3) having a set of buttons 508including dashboard button 518, document title 510, spreadsheet document512, spreadsheet table 514, and gadget 516 having gauges 520 and insertbutton 522.

In various embodiments, web applications, or “web apps,” may beintegrated with office applications via the instrumentality of plug-incomponent 306. Web applications are specialized applications configuredto run within a host program, such as Java applets that run within abrowser program. Thousands of web apps exist that perform many differenttasks including games, computations, different utilities, displays forvarious types of data, and the like. Mini web applications, such asGoogle Gadgets, SharePoint Web parts, portlets and other hosted webapplications are designed to live in a web browser and primarily extendweb based functions. For example, Google Gadgets are mini webapplications that were designed also to extend the functionality ofGoogle Docs by third party developers. Plug-in component 506 enables theexecution of web apps within the office applications, with or withoutsourcing the data from the office application. For example, a gadget mayintegrate with a spreadsheet to read data from the spreadsheet andvisualize the data in charts, pivot tables, and the like. In a typicalscenario, a user of spreadsheet 500 may search the web and find a dollargauge web app embodied in gadget 516, and insert the web app, usinginsert button 522, into table 514 to display dollar amounts in agraphical format (as opposed to a numerical format as is commonly usedin tables) within table 514. Gadget 516 may source the data, dependingon its settings and/or its original type, it displays from a localoffice file or from a remote server and graphically display the sourceddata.

FIG. 6 shows the example integrated document processing interface ofFIG. 3 including a dashboard with metadata. In various embodiments, theintegrated word processing interface 600 includes a title 602, a set ofmenus 604, a plug-in component 606 having a set of buttons 608 foractivation of various functions including a dashboard button 618, adocument 612 with a title 610, text 614, and multimedia data 616.Additionally, upon activation of dashboard button 618, a dashboard panel620 is presented having different metadata presentation areas such asfile information 622, author information 624, presence information 626,and related files information 628.

Those skilled in the art will appreciate that many other types ofmetadata associated with document 612, the user, an associated project,and the like may be presented in the dashboard area, such as date ofcreation, participants in the project, contact information, history ofdocument changes, and the like. Furthermore, those skilled in the artwill appreciate that although a text and graphics document is discussedherein, the descriptions about dashboard are substantially applicable toother office applications such as spreadsheet, presentation, and otherdocument processing applications.

In various embodiments, metadata associated with the document beingauthored by a user may be retrieved from local and/or remote sources anddisplayed on the dashboard. Such metadata may be embedded in thedocument itself, be associated with document name/identification in adatabase, or otherwise be accessible. In various embodiments, filemetadata is read-only, while in other embodiments, file metadata isread/write and may be modified by the user with the appropriatepermissions. In still other embodiments, some portions of the metadatais read only, such as file creation date, while others, such as filepermissions may be modified.

File information 622 metadata may include filename, file size, format,creation date, change history, permissions settings, and the like. Suchinformation may typically be embedded in the document file itself and/orbe calculated from the document file data. In some embodiments, a “callhome” metadata may be added to the file that allows a locally storedfiled to know what the source of the file is on the server. Suchmetadata enable transparent and consistent co-authoring of the samedocument by multiple authors. The call-home metadata help maintain alogical connection to the server such that two authors from differentcomputers accessing the same file locally on their respective computers,perceive their respective local files as being the server copy of thesame file.

Author information 624 metadata may include name of a primary author ordocument owner, collaborators, edit history for each author, and thelike.

Presence information 626 metadata may include the names and contactinformation, such as email and title, of collaborating authors fordocument 612, and the current online presence status of each author. Forexample, if John Doe is currently online but others are not, then anicon or similar indicator, such as color, may display their presence andavailability accordingly.

Related files information 628 metadata may include a number ofsubfields, such as related subject, related project, and related personareas for displaying information about files with related subject matterto document 612, project information involving document 612 or one ofits users/authors, and people associated with document 612 or itsauthors, such as project managers, respectively. In various embodiments,automatic searches may be performed to find and display in the dashboardarea related files, related subjects, related projects, or relatedpeople associated with document 612.

In various embodiments, dashboard 620 may also be used to display realtime communications between multiple collaborators or coauthors.Furthermore, the metadata may be dynamically updated as various datachange, such as the size of file during editing. Still further, thedisplay of metadata may be dynamically changed to indicate variousconditions, such as alarms (for example, for unauthorized attempts toaccess the document), alerts (for example, project meeting time based onoffice calendar application data), and the like. The various conditionsmay be indicated via color changes, icon changes, and via use of othersimilar visual or audible indicators. Real time collaboration amongmultiple authors is further described below with respect to FIG. 7A.

FIG. 7A shows an example client-server computing environment withcommunication between the server and the dashboard on the client. Serverview 722 includes a master document 724 and a merge and communicationmodule 726. In various embodiments, the server is in communication withclient 700 via a computer network. Client view 702 includes documentcopy 704, and dashboard 706 including multiple fields as shown anddescribed previously. In various embodiments, the server devices anduser client devices described with respect to FIG. 7A are substantiallysimilar to those described with respect to FIG. 2.

In various embodiments, multiple clients and/or users connect with theserver to edit copies of a single document. A master copy of the singledocument exists on the server and copies of the master document aredownloaded to the client devices corresponding to the users. Thoseskilled in the art will appreciate that a “master copy” is generally amain version of a document to which all other copies and/or versions ofthe same document are subordinate. Generally, the master documentincludes all the latest approved and/or valid changes to the particulardocument and such changes take precedence over any changes in any of thesubordinate versions of the particular document. The multiple users maycommunicate with each other via the server, which relays theircommunications to the other users. The communications between the usersmay or may not be related to the document currently being edited. Thecomments appear in real time in the appropriate comment area ofdashboard 706 of each client device. Comments entered by the users ontotheir respective dashboard comment areas are similarly downloaded to theserver for relay to the other user's client devices. Comments from eachuser may be identified by various techniques such as user identifiers,different colors, different icons, avatars, and the like.

In various embodiments, the servers, and/or the service providers, whichprovide the servers, provide API for client devices to communicate withthe servers. Plug-in component 606 (see FIG. 6) on the client side mayinclude multiple APIs for communicating with different ones of multiplethird party and unrelated collaboration services, selectable by the userand/or by automatic detection of the type of the connectedserver/service. In the absence of plug-in component 606 and its abilityto communicate with different third party collaboration services andplatforms through different corresponding APIs, using multiple disparateand incompatible collaboration services may not be possible. A dedicatedserver-side software merge and communication module may run on theservers for communication with the plug-in components to implement adistributed document merge and client communication process, as furtherdescribed below. Those skilled in the art will appreciate that theserver side merge and communication module may be included in a singlemodule or be decomposed into multiple modules.

In various embodiments, local document copies on user client devices areindependently edited by respective users collaborating and coauthoringthe same document. Periodically, each client device communicates withthe server to query the status of the master document and determinewhether other client devices or users have made changes that are nowincluded in the master document. Changes included in the master documentmay be signified by timestamps, flags, hash functions, CRC (CyclicRedundancy Check), and other similar techniques for detecting and/orrecording changes to data or document contents.

Additionally, each client device communicates with the server thechanges made to the local copy of the document on each client deviceitself. In response, the server records such changes, for example, in atime-stamped database, and incorporates the changes made by each clientdevice and communicates to each client device the changes made by otherusers. If there are changes made to the master document by other users,each client device downloads such changes made to the master documentfrom the server to update each client device's own local copy. Thoseskilled in the art will appreciate that such communications between theclient and the server may be initiated by either side, the client or theserver, via the use of APIs, by actively querying the occurrence of anevent, such as a document change, or sending a message to inform theother side of the occurrence of the event. In some embodiments, on aparticular platform or document processing suite, such as Microsoft'sOffice suite, a merge feature of the particular document processingsuite may be used to perform the merge function once the document isdownloaded to the client computing device.

In the alternative, each client device downloads the whole updatedmaster document including changes from other clients/users and replacesits own local copy of the document with the updated master document. Insome embodiments, a file may be downloaded from the server to a localclient computing device by a user, be locally merged with acorresponding local file currently being worked on by a user, and thecombined result of the merge be shown to the user. This way, all localcopies of the document are dynamically synchronized and merged. Thistechnique allows local copies of the document on the client devices tomaintain consistency with each other and with the master copy of thedocument in substantially real time. In effect, each user working on theuser's local copy of the document sees all the changes made by otherusers who are simultaneously and remotely working on other copies of thesame document. The above embodiments implement a distributed documentmerge algorithm with no central control of the whole merging and updateprocess.

In various embodiments, the changes made by a second user to a localcopy of a particular document on the second user's client device arecommunicated, by the server, to a first user's client device and thenmerged with the first user's local copy of the particular document onthe client device of the first user, while in other embodiments, theactual document merge takes place on the server and then communicated tothe first user's client device.

In various embodiments, user and/or client devices coauthoring the samedocument may be assigned different priorities. The changes made by auser having a higher priority take precedence over changes made byanother user having a lower priority, if such changes create a conflict.In other embodiments, all users have the same priority and the changesmade last are included in the master document and then propagate to alllocal copies as described above. In the latter embodiments, in effect,timestamp of the changes determine the priority, with the latesttimestamp having the highest priority.

In various embodiments, the servers are provided by different serviceproviders. Each service provider may have a master copy of the documentand server a number of clients. Each client can also use the services ofa number of service providers. Consistency of multi-authored documentsmay be maintained among multiple master copies on the servers of thecorresponding multiple service providers by merging the documents asfurther described below with respect to FIGS. 8A and 8B.

In various embodiments, client devices may work off-line (not connectedto the servers via computer networks). Changes made to documents by theuser on the client copy of the document, are cached continuously in thebackground without user intervention and are later merged with changesmade by other users to their respective copies of the same document uponreconnection to the servers via computer networks, as described above.

The descriptions above with respect to FIG. 7A are substantiallysimilarly applicable to documents used by other office applications,such as spreadsheets.

FIG. 7B shows an example integrated document processing interface forslide presentation applications usable with the computing device of FIG.2. In various embodiments slide presentation application interface 750of document processing suite includes title 752, set of menus 754,plug-in component 756 having set of buttons 758 for activation ofvarious functions including dashboard button 768, slide presentationdocument 762 with title 760, text 764, and multimedia data 766. The setof menus, plug-in component, set of buttons and dashboard button havesubstantially the same functionality as described above with respect toFIG. 3.

In various embodiments, multiple users may collaborate as coauthors onslide presentation document 762 substantially as described above withrespect to FIG. 7A above. The merge algorithm of slides may differ fromother document types in some ways due to the relatively more graphicalnature of slides compared with other documents. For example, in a textdocument, changes to a word are relatively easy to detect and updatebecause a word is composed of letters which are uniquely encoded in acomputer system, for example, as ASCII (American Standard Code forInformation Interchange) code. Whereas, changes to a bar graph are moredifficult to detect, record, and merge. As such, in various embodiments,changes to a slide presentation document may be made at slide levelgranularity by replacing a whole slide (one page) of the slidepresentation document rather than a portion of the contents of theslide. In other embodiments, contents of a given slide may be dividedinto text portions, for example, bulleted comments, and graphicalnon-text portions, for example, charts, pictures, drawings, and thelike. In the latter embodiments, changes to the text portions may bemerged as described above with respect to FIG. 7A for text-orienteddocuments, while the graphical portions may be replaced whole in case ofany changes to the graphical portions. In yet other embodiments, changesto the graphical portions of the slide may be tracked and merged withother versions of the slide by image processing techniques. For example,if the graphical portion of the slide is a bitmap image, the changes maybe recorded on pixel level and merged with other versions of the slideaccordingly.

In various embodiments, at whole slide level granularity (inter-slide),if a first user and a second user are collaborating on a slidepresentation document, if the second user changes a particular slide,but the first user does not touch the particular slide, then theparticular slide in the first user's copy of the document isautomatically replaced by the second user's version of the particularslide. But, if the first user has also touched/changed the particularslide, then in some embodiments or by selected settings, the first useris given an opportunity, for example, via a dialog box, to elect whetherto replace the first user's version of the particular slide with thesecond user's version or not.

In various other embodiments, if both the first and the second user havemodified a particular slide, the first user is not given the option toselect which version of the particular slide to be used in the slidepresentation document. Instead, the two versions of the slide are mergedautomatically at inter-slide granularity (whole slide), or atintra-slide granularity (portions within slide) as further describedbelow.

At inter-slide granularity, in various embodiments, a priority may beassigned to users to specify which user's version of modifications takesprecedence over other users. In these embodiments, the version of theslide modified by the user with the highest priority replaces the otherversions modified by other lower priority users.

In various embodiments, the merging of slides is effected bycommunications between the client devices and the server substantiallyas described above with respect to FIG. 7A.

In various embodiments, slides in the slide presentation document areidentified by unique IDs, hash codes, and the like, generated after eachchange by each user to keep track of which slides in the document havebeen touched by which users. Each modified slide is merged with thecorresponding slide in each user's local copy of the slide presentationdocument according to the illustrative procedures and algorithmsdescribed above for slide merge.

In various embodiments, if both the first and the second user havemodified a particular slide, at intra-slide granularity (contents withina slide), textual and character based contents are merged as describedabove for text documents with respect to FIG. 7A, while graphicalcontents are merged by replacing the whole graphical content as a unit,similarly to replacing a whole slide as described above with respect toslide level granularity procedure. In other embodiments, changes to thegraphical portions of the slide may also be tracked and merging of thegraphical portions be performed with a granularity and scope that coverssubparts of the graphical portions. For example, if the graphicalportion of the slide is vector based, the changes may be recorded atvector level and merged with other versions of the slide by resolvingand merging vector properties, such as direction and length.

FIG. 8A shows an example system structure for document processing suiteapplications. In various embodiments, system structure 800 includesmultiple providers 802 interfaced with abstraction layer 806 viainterfaces 804. Abstraction layer 806 use document processing suite API808, which may in turn use services of operating system 810 running onhardware layer 812.

By using abstraction layer 806, service providers 802 are shielded fromimplementation details of lower layers, such as document processingsuite API 808 and underlying operating system 810. Thus, serviceproviders 802 may interact with any document processing suite, operatingsystem, and hardware platform that provide access to abstraction layerthrough which core system capabilities, such as document processingsuite API 808 and operating system 810 services, may be accessed.Abstraction layer 806 may provide different interfaces each of which iscompatible with the requirements of a different corresponding serviceprovider. With this structure, in various embodiments, plug-in componentfunctionalities described above with respect to FIGS. 3-7B may be usedwith many different service providers and document processing suites. Invarious embodiments, the plug-in component is used with a single serviceprovider and a single document processing suite, while in otherembodiments, the plug-in component is used with a multiple serviceprovider and multiple document processing suites simultaneously. In thelatter embodiments, a particular document may be synchronized and keptconsistent on multiple servers from multiple service providers, as wellas multiple clients, as further described below with respect to FIG. 8B.

FIG. 8B shows an example interaction between multiple service providersand clients usable for collaboration. In various embodiments, systemconfiguration 840 includes multiple service providers 842 through 844,multiple clients 846, 848, through 850, each coupled with variousservice providers 842 through 844 via various network connections 852.

Each of the service providers 842 through 844 may have a separate mastercopy of a particular document, which may be coauthored by two or more ofthe clients 846, 848, through 850. In this distributed mergeconfiguration, once any one of the client devices modifies the document,the changes to the document automatically and in real time arepropagated to all other client and servers by the distributed algorithm,as described above with respect to FIG. 7A. As an illustrative example,client 848 downloads a copy of document 1 master from server of serviceprovider 842 and makes some changes to the document. Next time thatclient 848 checks the server to determine changes to the document byother clients, client 848 also communicates the changes it made to thedocument. When other clients check the server for the same purpose, thechanges to the document previously made by client 848 now propagates tothe other clients. The same process is periodically repeated between allclients and all other service providers in a distributed manner,updating the other master copies of the document on other servers. Thus,all copies of the document, master copies and client copies, are updatedwith all changes from all collaborating clients in substantially realtime and all such copies of the document maintain consistency with othercopies.

FIG. 9 shows an example flow diagram for document processing. Routine900 proceeds to block 910 where a user associated with a client deviceis enabled to select and load onto the client computing device aparticular document using the plug-in component integrated interfacedescribed previously with respect to FIGS. 3-7B. The routine proceeds toblock 920.

At block 920, the user is enabled to select collaboration with otherusers/clients, for example, by selecting the Collaboration button in theset of buttons presented by the integrated interface of the plug-incomponent (for example, see FIG. 3). The routine proceeds to block 930.

At block 930, any changes made to the selected and loaded document issynchronized and merged with other document copies on othercollaborating clients, as described in detail above with respect to FIG.7A. Dashboard section of the plug-in component may be used for real timecommunications between the collaborating clients. The routine proceedsto block 940.

At block 940, various metadata associated with the selected documentand/or the user may be displayed on the dashboard. Such metadata mayinclude information about the document, information about othercollaborating authors, presence information showing which clients arecurrently working on the selected document, information about a projectinvolving the selected document, information about any related files,and the like. The routine proceeds to block 950.

At block 950, interface is provided for the user to perform image searchand web search and insert portions of the results into the document, asdescribed earlier with respect to FIGS. 4A and 4B. The routine proceedsto block 960.

At block 960, the routine terminates.

FIG. 10 shows an example flow diagram for a word processing applicationmerge routine. Routine 1000 proceeds to block 1010 where changes to alocal copy of a selected document are made by a first user. Withadditional reference to FIG. 7A, the local copy of the selected documentresides on a client device associated with the first user, which is incommunication with a server of a service provider. Multiple clientdevices and corresponding users may be in communication with the serverwhile collaborating with the first user on the selected document. Invarious embodiments, the changes made to the local document copy aredetected by the plug-in component running on the client device and areperiodically communicated to a merge and communication module running onthe server, as more fully described above with respect to FIG. 7A. Theroutine proceeds to block 1020.

At block 1020, a master copy of the selected document on the server isupdated with the changes made by the first user and communicated to theserver. In various embodiments, the changes to the document may berecorded as differences from the original version of the document, whilein other embodiments, the whole document may be replaced by a newerversion to effect such changes. The routine proceeds to block 1030.

At block 1030, the client device associated with the first user checksthe server to ascertain whether a second user has modified a second copyof the selected document on a client device associated with the seconduser. Such information about changes made to the document by the seconduser are communicated to the client device of the first user via themerge and communication module of the server. The routine proceeds todecision block 1040.

At decision block 1040, if new changes are made to the selected documentby the second user, the routine proceeds to block 1050. Otherwise, theroutine proceeds back to block 1010.

At block 1050, the local copy of the selected document on client deviceof the first user is updated to include the changes made by the seconduser. In various embodiments, the changes made by the second users arecommunicated, by the server, to the first user's client device and thenmerged with the first user's document on the client device of the firstuser, while in other embodiments, the actual document merge takes placeon the server and then communicated to the first user's client device.The routine proceeds to block 1060.

At block 1060, the routine terminates.

FIG. 11 shows an example flow diagram for a spreadsheet applicationmerge routine. Routine 1100 is substantially similar to routine 1000described above with respect to FIG. 10. The routine proceeds to block1110 where changes to a local copy of a selected spreadsheet documentare made by a first user. With additional reference to FIG. 7A, thelocal copy of the selected document resides on a client deviceassociated with the first user, which is in communication with a serverof a service provider. Multiple client devices and corresponding usersmay be in communication with the server while collaborating with thefirst user on the selected document. In various embodiments, the changesmade to the local document copy are detected by the plug-in componentrunning on the client device and are periodically communicated to amerge and communication module running on the server, as more fullydescribed above with respect to FIG. 7A. The routine proceeds to block1120.

At block 1120, a master copy of the selected document on the server isupdated with the changes made by the first user and communicated to theserver. In various embodiments, the changes to the document may berecorded as differences from the original version of the document, whilein other embodiments, the whole document may be replaced by a newerversion to effect such changes. The routine proceeds to block 1130.

At block 1130, the client device associated with the first user checksthe server to ascertain whether a second user has modified a second copyof the selected document on a client device associated with the seconduser. Such information about changes made to the document by the seconduser are communicated to the client device of the first user via themerge and communication module of the server. The routine proceeds todecision block 1140.

At decision block 1140, if new changes are made to the selected documentby the second user, the routine proceeds to block 1150. Otherwise, theroutine proceeds back to block 1110.

At block 1150, the local copy of the selected document on client deviceof the first user is updated to include the changes made by the seconduser. In various embodiments, the changes made by the second users arecommunicated, by the server, to the first user's client device and thenmerged with the first user's document on the client device of the firstuser, while in other embodiments, the actual document merge takes placeon the server and then communicated to the first user's client device.The routine proceeds to block 1160.

At block 1160, the routine terminates.

FIG. 12 shows an example flow diagram for a slide presentationapplication merge routine. With additional reference to FIG. 7B, routine1200 proceeds to block 1210 where slides in a slide presentationdocument are monitored for changes. Each of a first and a second clientdevice, associated with a first and a second user, respectively, has alocal copy of the slide presentation document and is in communicationwith a server operated by a service provider. The server has a mastercopy of the slide presentation document. All copies of the slidepresentation document on the client devices and the server are updatedperiodically to stay consistent with each other as further describedbelow. The routine proceeds to decision block 1220.

At decision block 1220, it is ascertained whether a new slide was addedor changes were made to an existing slide by only the second user andnot the first user. If so, the routine proceeds to block 1230.Otherwise, the routine proceeds to decision block 1240.

At block 1230, the slide added or changed by the second user is added orreplaces the corresponding slide in the first user's copy of thedocument. Such merge and/or update of the slide is performed incommunication with the server as described above at least with respectto FIG. 7B.

At decision block 1240, the first user has also modified the slide. Atthis block, it is ascertained whether the first should be queried tospecify which copy of the slide to use, the copy modified by the firstuser or the copy modified by the second user. If the first user is tospecify the version of the slide to be used in the copy of the documentassociated with the first user, then the routine proceeds to block 1250,otherwise, the routine proceeds to block 1260.

At block 1250, the first user is queried as to which copy/version of themodified slide is placed in the slide presentation document copyassociated with the first user, the version as modified by the firstuser or the version modified by the second user. Accordingly, theappropriate version is placed in the copy of slide presentation documentof the first user. The routine proceeds to block 1270.

At block 1260, both the first and the second user have modified the sameslide, but the first user does not have the option to select which slideversion is to be used. In this case, the slides are merged automaticallyto resolve content differences as described above with respect to FIG.7B. Specifically, in embodiments with inter-slide granularity, users maybe assigned different priorities and the version of the slide modifiedby the user with the highest priority replaces all other users'versions. In embodiments with intra-slide granularity, the contents ofthe different slides are merged in the text and the graphical portionsof the slide as described above with respect to FIG. 7B. The routineproceeds to block 1270.

At block 1270, the routine terminates.

It will be understood that each block of the flowchart illustration, andcombinations of blocks in the flowchart illustration, can be implementedby computer program instructions. These program instructions may beprovided to a processor to produce a machine, such that theinstructions, which execute on the processor, create means forimplementing the actions specified in the flowchart block or blocks. Thecomputer program instructions may be executed by a processor to cause aseries of operational steps to be performed by the processor to producea computer implemented process such that the instructions, which executeon the processor to provide steps for implementing the actions specifiedin the flowchart block or blocks. The computer program instructions mayalso cause at least some of the operational steps shown in the blocks ofthe flowchart to be performed in parallel. Moreover, some of the stepsmay also be performed across more than one processor, such as mightarise in a multi-processor computer system. In addition, one or moreblocks or combinations of blocks in the flowchart illustration may alsobe performed concurrently with other blocks or combinations of blocks,or even in a different sequence than illustrated without departing fromthe scope or spirit of the disclosure.

Accordingly, blocks of the flowchart illustration support combinationsof means for performing the specified actions, combinations of steps forperforming the specified actions and program instruction means forperforming the specified actions. It will also be understood that eachblock of the flowchart illustration, and combinations of blocks in theflowchart illustration, can be implemented by special purpose hardwarebased systems which perform the specified actions or steps, orcombinations of special purpose hardware and computer instructions.

Changes can be made to the claimed invention in light of the aboveDetailed Description. While the above description details certainembodiments of the invention and describes the best mode contemplated,no matter how detailed the above appears in text, the claimed inventioncan be practiced in many ways. Details of the system may varyconsiderably in its implementation details, while still beingencompassed by the claimed invention disclosed herein.

Particular terminology used when describing certain features or aspectsof the disclosure should not be taken to imply that the terminology isbeing redefined herein to be restricted to any specific characteristics,features, or aspects of the disclosure with which that terminology isassociated. In general, the terms used in the following claims shouldnot be construed to limit the claimed invention to the specificembodiments disclosed in the specification, unless the above DetailedDescription section explicitly defines such terms. Accordingly, theactual scope of the claimed invention encompasses not only the disclosedembodiments, but also all equivalent ways of practicing or implementingthe claimed invention.

The above specification, examples, and data provide a completedescription of the manufacture and use of the claimed invention. Sincemany embodiments of the claimed invention can be made without departingfrom the spirit and scope of the disclosure, the invention resides inthe claims hereinafter appended. It is further understood that thisdisclosure is not limited to the disclosed embodiments, but is intendedto cover various arrangements included within the spirit and scope ofthe broadest interpretation so as to encompass all such modificationsand equivalent arrangements.

What is claimed is:
 1. A method of co-authoring a document, the methodcomprising: detecting a first set of changes made to a first version ofa particular document on a first client computing device; detecting asecond set of changes made to a second version of the particulardocument on a second client computing device; and communicating everychange in the first set of changes, using a first software plug-incomponent plugged into a first existing document processing suite, tothe second client computing device, and communicating every change inthe second set of changes, using a second software plug-in componentplugged into a second existing document processing suite, to the firstclient computing device via a plurality of third party collaborationservice providers unassociated with the first and the second documentprocessing suites used to edit the particular document, each of theplurality of the third party collaboration service providers supplying aseparate master copy of the particular document identical to each of theseparate master copies of the particular document on each of the otherplurality of the third party collaboration service providers, resultingin the first version and the second version of the particular documentto become identical, and also resulting in all separate master copies ofthe particular document to become identical; and wherein the first andthe second software plug-in components are configured to provide anintegrated user interface including image search, copy, and insertfunctions, the search function configured to be conducted usingthird-party external search engines coupled with the plug-in components.2. The method of claim 1, further comprising using a first type ofdocument processing suite to make the first set of changes and using asecond type of document processing suite to make the second set ofchanges.
 3. The method of claim 1, further comprising displayingmetadata related to the particular document on a dashboard area providedby a plug-in component, associated with a document processing suite usedto edit the particular document, on at least one of the first and thesecond client computing devices.
 4. The method of claim 3, whereindisplaying metadata related to the particular document on the dashboardarea comprises displaying metadata according to a permission associatedwith at least one of the first and the second client computing devices.5. The method of claim 3, wherein displaying metadata related to theparticular document on the dashboard area comprises displaying metadataaccording to a selection of metadata by a user of the at least one ofthe first and the second client computing devices.
 6. The method ofclaim 3, wherein the metadata includes call-home comprising a locationof an original copy of the particular document on the third partycollaboration service provider.
 7. The method of claim 3, wherein themetadata is obtained using a dynamic discovery process.
 8. A firstclient computing device executing a software plug-in component by aprocessor, the software plug-in component configured to be plugged intoand integrated with an existing document processing suite to: present anintegrated user interface within the document processing suite, theintegrated user interface configured to communicate with a plurality ofthird party collaborative service providers not associated with thedocument processing suite, to communicate all changes made in differentversions of a particular document during editing of the differentversions of the particular document, using the software plug-incomponent, each of the plurality of the third party collaborationservice providers supplying a separate master copy of the particulardocument identical to each of the separate master copies of theparticular document on each of the other plurality of the third partycollaboration service providers, resulting in all versions of theparticular document to be identical, and also resulting in all separatemaster copies of the particular document to become identical; andwherein the software plug-in component is further configured to providean integrated user interface including image search, copy, and insertfunctions, the search function configured to be conducted usingthird-party external search engines coupled with the plug-in component.9. The first client computing device of claim 8, further comprising adashboard area configured to display metadata about a particulardocument edited using the document processing suite.
 10. The firstclient computing device of claim 9, wherein the metadata includes alocation of an original version of a particular document on a server ofeach of the plurality of the third party collaboration serviceproviders.
 11. The first client computing device of claim 9, wherein themetadata displayed in the dashboard area is displayed according tocorresponding preset permissions for the metadata.
 12. The first clientcomputing device of claim 9, wherein the metadata is automaticallyobtained via a discovery process by the plug-in component.
 13. The firstclient computing device of claim 8, wherein the integrated userinterface is further configured to simultaneously communicate withmultiple third party collaboration service providers.
 14. The firstclient computing device of claim 13, further causing the first clientcomputing device to allow a user of the first client computing device toremotely set permissions associated with a particular document on aserver of the plurality of the third party collaboration serviceproviders.